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, After a recollection on compression through a projection onto a polyhe- 

dral set (which generalizes the compression by coordinates quantization), 
we express, in this framework, the probability that an image is coded with 
K coefficients as an explicit function of the approximation error. 

> , 

m . 1 Introduction 

(N 

In the past twenty years, many image processing tasks have been addressed 
with two distinct mathematical tools : Image decomposition in a basis and 
[ optimization. 

■ The first mathematical approach proved very useful and is supported by 
solid theoretical foundations which guarantee its efficiency as long as the basis 
is adapted to the information contained in images. Modeling the image content 

^ _ by appropriate function spaces (of infinite dimension), mathematical theorems 

S ■ tell us how the coordinates of an image in a given basis behave. As instances, 

^1 it is possible to characterize Besov spaces (see ^J) and the space of bounded 

variation (it is "almost characterized" in [3]) with wavelet coefficients. As a 
consequence of these characterizations, one can obtain performance estimate 

■ for practical algorithms (see Th 9.6, pp. 386, in ^U] and [S]^ for more complex 
analyses) . Image compression and restoration are the typical applications where 
such analyses are meaningful. 

The optimization methods which have been considered to solve those prac- 
tical problems also proved very efficient (see I13j . for a very famous example). 
However, the theory is not able to assess how well they perform, given an image 
model. 
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Interestingly, most of the community who was primarily involved in the 
image decomposition approaches is now focusing on optimization models (see, 
for instance, the work on Basis Pursuit [2] or compressed sensing The main 
reason for that is probably that optimization provides a more general framework 

(PITIIHI). 

The framework which seems to allow both a good flexibility for practical 
applications (see |2] and other papers on Basis Pursuit) and good properties 
for theoretical analysis is the projection onto polyhedron or polytopes. For the- 
oretical studies, it shares simple geometrical properties with the usual image 
decomposition models (see 0). This might allow the derivation of approxima- 
tion results. 

The aim of this paper is to state a rigorous^ theorem which relates, asymp- 
totically as the precision grows, the approximation error and the number of 
coefficients which are coded (which we abusively call codelength, for simplic- 
ity). More precisely, when the initial datum is assumed random in a convex set, 
we give the probability that the datum is coded by K coefficients, as a function 
of the approximation error (see theorem 13. II for details). 

This result is given in a framework which generalizes the usual coding of 
the quantized coefficients ( "non- linear approximation" ) , as usually performed 
by compression standards (for instance, JPEG and JPEG2000). 



2 Recollection on variational compression 

Here and all along the paper is a non- negative integer, / — {1, . . . ,iV} and 
B = {tpi)i^i is a basis of M.^ . We will also denote for r > (all along the paper 
T denotes a non-negative real number) and for all k ^ Z, = T{k — i). 
For any {ki)i^j G Z^, we denote 



C((fci)ig/) = |2^MjV'i,Vi e /, Tfc, <Ui < Tk,+i I . (1) 
We then consider the optimization problem 



minimize f{v) 

under the constraint v E C {{ki)i£i) , 

where / is a norm, is continuously differentiable away from and its level sets 
are strictly convex. In order to state Theorem l3.1l we also need / to be curved. 
This means that the inverse of the homeomorphism'^ h below is Lipschitz. 

/.:{uGM^, /(«)-!} ^ {.geM^, ||ff|l2 = l} 

v/(") 



u 



llV/(u) 



^The theorem concerning compression in |5| is false. The situation turns out to be more 
complex than we thought at the time it was written. 

■^We prove in |5] that, under the above hypotheses, h actually is an homeomorphism. 
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(The notation ||.||2 refers to the eucUdean norm in R^.) 
We denote, for any [kiji^i S Z^, 

J iih)iei) = {« e = Tk, or u* = Tk^+i}, 

where u* = ^i^jV-ii'i is the solution to (P) {{ki)i^j). 

The interest for these optimization problems comes from the fact that, as 
explained in we can recover {ki)i^j from the knowledge of (J, {u*)j^j) (where 
J=Ji{h),ei)). 

The problem (P) can therefore be used for compression. Given a datum 
u = ^i^jUitpi e M^, we consider the unique {ki{u))i^i G such that (for 
instance) 

V« G /, Tk^(u) < U.-L < Tk^(u) + 1- (2) 

The information (J, (u*)jgj), where J = J {{ki{u))i^i), is then used to encode 
u. In the remainder, we denote the set of indexes that need to be coded to 
describe u by J{u) = J {{ki{u))i^i). 

Notice that we can also show (see 0) that the coding performed by the 
standard image processing compression algorithms (JPEG and JPEG2000) cor- 
responds to the above model when, for instance, 

/(5]t.,^.) = 5]Kp. 

3 The estimate 

The theorem takes the form : 

Theorem 3.1 Let r' > and U be a random variable whose low is uniform 
in Cf^ [t'): for a norm fd- Assume f satisfies the hypotheses given in Section 
For any norm \\.\\ and any K G {1, . . . N} there exists Dk such that for all 
£ > 0, there exists T > such that for all t < T 

P J ([/) = < DkE^ + e, 
where E is the approximation error^ : 

e = k(^\u -TY,h{u)^^\\^ . 

Moreover, if fiY.i^i'^i^i) = Y^iei '^^ have'^ 

# J ([/) = k] > DkE^ttt - £, 

^When computing the approximation error, we consider the center of C {(A;i)ig/) has been 
chosen to represent all the elements such that (fci)ig/ = iki{u))i^j. 

■'This assumption is very pesimistic. For instance, the lower bound seems to hold for almost 
every basis B of R^, when / is fixed. We have not worked the details of the proof of such a 
statement out though. 
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When compared to the kind of results evoked in Section^ the above theorem 
original is in several ways : 

First, it concerns variational models which are more general than the model 
in which the results of Section are usually stated. This is probably the main 
interest of the current result. For instance, by following reasonings similar to 
those which led to Theorem l3.1l it is probably possible to obtain approximation 
results with redundant transforms. 

Secondly, it expresses the distribution of the number of coefficients as a func- 
tion of the approximation error, while former results do the opposite. Typically, 
they bound the approximation error (quantified by the norm) by a function 
of the number of coefficients that are coded. The comparative advantages and 
drawbacks of the two kind of statements is not very clear. In the framework of 
Theorem l3.1l the larger Dk (for K small), the better the model compresses the 
data. However, it is clear that, as the approximation error goes to 0, we have 
more and more chances to obtain a code of size N . With this regard, the con- 
stant Dk-i seems to play a particular role since it dominates (asymptotically 
as r goes to 0) the probability not to obtain a code of length N. 

Thirdly, it is stated in finite dimension and, as a consequence, it does not 
impose apriori links between the data distribution (the function J^) and the 
model (the function / and the basis B). The ability of the model to represent 
the data is always defined. For instance, this allows the comparison of two bad 
models (which is not possible in infinite dimension). The analog of Theorem l3.1l 
in infinite dimension might be interesting, though. 
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4 Proof of Theorem 13.11 

4.1 First properties and recollection 
4.1.1 Rewriting (P) 

For any u G M^, {P){u) denotes the optimization problem 



(P) (u) 



minimize f{v — u) 

under the constraint w G C (0) . 



where denotes the origin in and C(.) is defined by CEJ. 
We then denote, for any u ~ J2iei '^'■'^^ G C (0), 

T T 

J{u) = {i e I,Ui = - or Ui = --}. 

With this notation, the set of active constraints of the solution u* to (P) {u) is 
simply J{u*). 

Proposition 4.1 For any (fci)ig/ G 

J{{h),ei) = J{u*), 
where u* is the solution to (P) {t '^^^j kit/jij . 

Proof. Denoting u* the solution of (P) {{ki)i^j) and u* the solution to (P) (r X^ie/ ^iV'i) j 
we have 

M* = U* + T ^ fcj-f/)!. (3) 

This can be seen from the fact that (P) ^i'^i) exactly (P) {{ki)i^i), 

modulo a "global translation" by (The rigorous proof of (0) can 

easily be established using Kuhn- Tucker conditions, see Th 28.3, pp. 281.) 

The proposition is then obtained by identifying the coordinates of u* and u* 
in the basis B. □ 



4.1.2 On projection onto polytopes 

We can now adapt the definitions and notations of ^ to the problems (P)(.). 
Beside Proposition 14.61 all the results stated in this section are proved in 0. 

We consider a norm fd (which will be used latter on to define the data 
distribution law) and define for any C C and any A C M 

S^ = {ue M^, 3u* G C,it* is solution to (P)(u) and fd{u - u*) G A] . 

This corresponds to all the optimization problems whose solution is in C (we 
also control the distance between u and the result of {P){u)). Notice that 
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depends on r. We do not make this dependence explicit since it does not create 
any confusion, in practice. 

We also define the equivalence relationship over C(0) 

For any u G C(0), we denote u the equivalence class of u. 

In the context of this paper, we obviously have for all u — J^i&i iJ-i''Pi G C(0) 

u=L^ + T J2 ^ Jiu), _i < < U , (4) 

[ j^JM J 

where 

(Here and all along the paper the notation j ^ J stands for j £ I \J.) 
Let us give some descriptions of 5'. 

Proposition 4.2 For any u* G dC{0) and any v £ u* , 

In words, Sl is a translation of 5^* . 

Proposition 4.3 For any u* £ dC{Q), any v £ and any A > 

u + X{v - u ) e 6„. 
Theorem 4.4 For any u* G dC{0) and any t' > 0, 

= {y + x(^u-u*)Jorv e vF,Xe]0,T'] and u e S^.} 
We also have (see |2]) 

Proposition 4.5 /// satisfies the hypotheses given in Section\^ for any u* G 

dC{0), S^t is a non-empty, compact Lipschitz manifold of dimension ^J{u*) — 1. 

Another useful result for the purpose of this paper is the following. 

Proposition 4.6 /// satisfies the hypotheses given in Section\^ for any u* G 

dC{0) and any t' > 0, S^°f'^ is a non-empty, bounded Lipschitz manifold of 
dimension ^J{u*). 

Proof In order to prove the proposition, we consider u* = J^iei & dC{0) 
and = J2i£j{u--') '^ii^i- We are going to prove the proposition in the particular 
case where u'^ — u* . Proposition 14.21 and 14.31 permit indeed to generalize the 
latter result obtained to any S^^^'^ \ for u* G u^. (They indeed guarantee that 
iS|f.'^ ' is obtained by translating iSj^l'^ 
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In order to prove that s}^^'^ ' is a bounded Lipschitz manifold of dimension 
^J{u*), we prove that the mapping h' defined below is a Lipschitz homeomor- 
phism. 

h': 5i.x]0,r'] ^ S^^'^ 

{u,X) I — > u= + A(m~w=). ^ ' 

The conclusion then directly follows from Proposition 14.51 

Notice first that we can deduce from Proposition 14.31 that h' is properly 
defined. 

Let us prove that h' is invertible. For this purpose, we consider Ai and A2 
in ]0,t'] and ui and U2 in S^c such that 

u" + Xi{ui - u'') ^ u"" + X2{U2 - u''). (6) 

We have 

Ai = /d(Ai(ui - w^)) 
- A2. 

Using lO, we also obtain ui — U2 and h' is invertible. 

Finally, h' is Lipschitz since, for any Ai and A2 in ]0,t'] and any ui and U2 
in 5^c, 

||Ai(ui-m'=)-A2(u2-u")||2 = ||Ai(ui-U2) + (Al-A2)(u2-u')||2, 

< T'||lii-U2||2+C|Ai-A2|, 

where C is such that for all u G , 

\\U-U-\\2<C. 

(Remember S^^a is compact, see Proposition ^3]) □ 



4.2 The estimate 

We denote the discrete grid by 

iG/ 

and, for u* £ 9C(0) and G Z'-'f"*), 

^((fcj)jeJ(«-)) = 51 where G Z^^''^"*^}. 

jeJ{u') i^J{u') 

The set P {ikj)j^j{u*)) is a slice in V. 
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Proposition 4.7 Let t' > 0, u* £ dC{0) and {kj)j(zj(^u') G 

nP((fc, <1. 

Proof. Taking the notations ofthe proposition and assuming sl^^ 'nX> 
0, we consider (fcj^)ie/ and {kf)i^i such that 

T^fci^, e5J^^'lnP((%),ej(«-)) 



and 



Theorem 14.41 guarantees there exist vi and V2 in u*, Ai and A2 in ]0,t'] and ui 
and W2 in S^ * such that 



and 



fcjVi = wi + Ai(ui - u*) 

T^^kfl/Ji = ?72 + A2(?i2 - U*). 



So 

Vi + Xi{ui - U*) ^ V2 + \2{U2 - U*) + T ^ [k] - k'^)'lj}i. 

Using we know there exists {Pi)i^j{u-') and {0i)i(^j{u*) such that 
^ J(7i*), < p] < i and - i < /3f < ^ 

= + T ^ /3,Vi 

and 

with = EjGj(n*) ^ijV'j' where u* = J^iei K^i- 

So, letting for all i ^ J{u*), ai — k] ~ kf + (3f — (3}, we finally have 

Ai(ui - u*) = A2(?i2 - u*) + T ^ aitpi. (7) 

i^,/(u*) 



Let us assume 



max I a, I >0, (8) 

J(«*) 
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and consider < A < 1 such that 



A < 7: r-r- (9) 

2maxj^j(„.) \ai\ 



We have, using Q, 



u" + XXi[{ui - u* + u") - u"] = u" + \\i{ui ~ u*) 

= u'^ + \t ailpi + XX2{u2 — u*) 

= V + AA2 [(U2 — U* + v) - v], 

where v = u'^ + XTj2i^j(u') ^ii^i- Moreover, using Q and ©, we know that 
V ^ u'^. Using Proposition we know that 

Ml — u* + G and U2 — u* + f G 5^. 

Finally, applying Theorem 14.41 we obtain 

+ AAi(mi ~u*)e S^°/^ n 

Since the solution to (P)(u'^ + AAi(iti— u*)) is unique, we necessarily have u'^ = v 
and therefore maxi^j(„.) jail = 0. This contradicts © and guarantees that 

max \ai\ = 0. 

Using the definition of a^, we obtain, for all i ^ J{u*), 

\kl - kf\ ^ \Pl ~ f3f\ < I. 
This implies kj — kf, for alH G /. □ 
Let us denote, for u* G dC{0), the projection onto Span{il;j,j G J{u*)) by 
p: — > Spani^PjJ e J{u*)) 

It is not difficult to see that, for any t' > 0, m* G dC{0) and {kj)j^,ji^u') G 

#(5gi^'lnP((fc,),ey(„.)))=l^r ^ A:,^,.Gp(5gi^'l). (10) 

j(iJ(u') 

Remark 1 Notice that the converse implication does not hold in general. It is in- 
deed possible to build counter examples where ' passes between the points of 
the discrete grid T>. However, it is not difficult to see that, */ 'T' X]j6j(u*) ^j^j ^ 
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p (5]^^'') and nP = 0, we can build {ki)i^j(u*) £ Z'A-'C"*) 

such that 

j6J(m*) i^J(u') 

where 

u'^ = u*il)j.{u* are the coordinates of u*) 

This means that the set S^^c'^ ' , which is a manifold of dimension ^J{u'^) living 
in M^, intersects a discrete grid. This is obviously a very rare event. Typically, 
adding to the basis B some kind of randomness (for instance adding a very small 
Gaussian noise to every tpi) would make it an event of probability 0. 

Notice, with this regard, that when f{J2i£i "^i^i) = Sis/ 1*^*1^' ''^^ trivially 
have the equivalence in 

A simple consequence of (|1(J|I is that 



V I j6./(«-) 



(11) 

Notice finally that, for u* — J2i£i '^i'^i ^ •9^(0), Proposition 14.21 and Equation 
(0} guarantees that 

for w" = Eje./(«-)"iV'i- 

We therefore have, using also Theorem 14.41 Proposition 14.31 and Equation 

Q, 

P ('^5"'') = {^^(«) + ^(?'(") - f""^ e ^, A e]0, r'] and u G 5^. }, 

= {u" + Xip{u) - u"), for A e]0, r'] and u e 5^^}, 

Finally, 

(12) 

Proposition 4.8 /// satisfies the hypotheses given in Section\^then, for any 

u* = J^iei'^i'^i ^ dC{0), p(s}^a''^^ (where u" = I]je./(n*) V'i j is a non- 
empty, bounded Lipschitz manifold of dimension ^J{u*). 
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Proof. Thanks to Proposition it suffices to establish that the restriction of 

u I — > p{u). 

is a Lipschitz homeomorphism. This latter result is immediate once we have 
established that p' is invertible. 

This proof is similar to the one of Proposition 14. 71 Taking the notations of 
the proposition, we assume that there exist ui and U2 in s}^c'^ ' and G 
R-'("*) satisfying 

Ui =U2+T ^ ailpi. 

If we assume maxj^jf^.-j \ai\ ^ 0, we have for < A < min(l, 2max gj( .) | a | )' 
u° + A(ui - u^) = u'^ + r ^ Aaii/ii + A(u2 - w^) 

= f + A j M2 + T AckiV'j ~ ''^ 

for V = w'^ + T^-^jj^.j Xait/^i. Since v G u'^ (see Q), Proposition l4 . 21 eruarantees 

that it2 + r X]ig j(M*) ^ctitpi = U2 + V — u"^ ^ S^v'^^- As a consequence, applying 
Proposition 14.31 we know that 

Since {P){vf + \{ui — u'^)) has a unique solution, we obtain a contradiction 
and can conclude that for all i ^ J{u*), maxi^j(„.) \ai\ = 0. 

As a consequence, p' is invertible. It is then obviously a Lipschitz homeo- 
morphism. □ 



Proposition l4.8l guarantees that p yS!^o^ ' j is Lebesgue measurable in ]R#'^(" ' . 
Moreover, its Lebesgue measure in R'^'^'" ' (denoted L^j(„.) (^p (^s}^a^ '^^) is 



< oo. 



finite and strictly positive : 

< L#,,(„.) (p (Si'/^ 

Another consequence takes the form of the following proposition. 
Proposition 4.9 Let t' > and u* £ dC{0) 

lim.-#(5r'np)<L.(,(5ri)) 

where K = ^J{u*). 
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Moreover, if the equality holds in ( or equivalently : the equality holds 
in ifUl) ] 

limr^-#(5i^^'lnp)=L^(p(5r'l 
Proof. In order to prove the proposition, we arc going to prove that, denoting 



The conclusion follows from (|12|l . 



(13) 



Let us first remark that, unhke s]^a'^ \ the set 

does not depend on r. This is due to Proposition 9^, in [^. Notice also that, 
because of Proposition both A and p ^<s|fc'^ are Lebesgue measurable (in 
K^) and that 

(A)-L;,(p(5l"/1)). 

In order to prove the upper bound in 113(1 . we consider the sequence of 
ftinctions, defined over 

fniu) — max 0, 1 — n inf llu — ulU 

This is a sequence of functions which are both Lebesgue and Riemann inte- 
grable and the sequence converges in L^(M^) to 1a (the indicator function of 
the set A). So, for any e > 0, there exists n G N such that 

fn< j ^A+e- 
Moreover, we have, for all u G and all n G N, 



^a{u) < fniu). 



^Tho definition of Sq given in the current paper does not allow the rewriting of the 
proposition 9 of ,9j. This is why we have not adapted it in Section 14.1.21 
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So, denoting Vr = [TY^jeJiw) ~ u", (fcj)jej(«c) S Z"'^"')!, 

lim [p(st/^) n I r ^ (fc,),eJ(„e) € Z'^("^) I = lim ^ l^H 



r— »-0 -"^^ — ^ 

< I J, 



< 



So, 



The lower bound in is obtained in a similar way, by considering an ap- 
proximation of 1a by a function smaller than 1^ which is Riemann integrable. 
(For instance : /n(u) = 1 — max(0, 1 — n inf.„^A ||m — w||2))-) D 



From now on , we will denote for all /v G {1, . . . , N} 
Ck ~ S where J C I, #J = K and Vj G J, 7ij = — or = — 

The set Ck contains all the " centers" of the equivalence classes of codimcnsion 
K. 

Similarly, we denote 

CIk = K e dC{0), #J{u*) = K} . 
We obviously have, for all K G {1, . . . , N}, 

CIk = ^weCkU''- 

Since, for all if G {!,..., N}, Ck is finite, it is clear from Proposition 14.91 
that, for any r' > 0, 

hm r-# (Slrj' nv)< J2^k{p (siY^)) < +^ 

Moreover, we have an equality between the above two terms, as soon as the 
equality holds in l(TT|) . 

We can finally express the following estimate. 
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Proposition 4.10 Let r' > 



where K — ^J{u*). 

Moreover, if the equality holds in for all u'^ G Ck ( or equivalently : the 
equality holds in H12(l ) 



Proof. We consider 



M = sup fd{u) 

We have, for all u* G dC{0), 

fd{u*) < Mt. (14) 
We therefore have for all u G (t') and for u* the solution to (P)(u), 

< t' + Mt. 

So 

Moreover, it is not difhcult to see that (remember h' defined by © is an home- 
omorphism) 



We can therefore deduce (from Proposition that 

hm (4°;^[ U Cf, (r') fl p) < J] L,, [p [5^' 



In order to prove the last statement of the proposition, we consider u* G 
9C(0) and u G S^".'"''\ we know that 

fdiu) < fd{u-u*) + fdiu*) 
< t' + Mt 



So 

4]r'"'c4;fin/:,,(r'). 
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Since (again) 

we know that the second statement of the proposition holds. □ 



Another immediate result is useful to state the final theorem. Notice first 
that we have, for any (fci)^^/ G and any norm ||.||, 

\\v~Ty^hM\dv^CT^+\ 

veC{(k,).ei) i^i 

where 



C= / \\v\\dv 
J{v=Y:,^iV,i,..yiei.\vi\<^} 

only depends on the particular norm ||.|| and the basis {il'i)iei. 

So, denoting U a random variable whose law is uniform in Cf^ (r') and 
{ki{U))i^i the discrete point defined by (0), we have 

lim^^^^^l^flM^.C. (15) 

This follows from the fact that the number of points (fci)ig/ such that 
C{{ki)i^i) intersects both Cf^ (t') and its complement in becomes negligible 
with regard to the number of points {ki)i^i such that C{{ki)i^i) is included in 
Cf^ (t'), when t goes to 0. 

We can now state the final result. 

Theorem 4.11 Let t' > and U be a random variable whose low is uniform in 
Cf^ (t'), for a norm fd- For any norm \\.\\, any K G {1, . . . N} and any £ > 0, 
there exists T > such that for all t < T 

^# J (U) = K^< DkE^ + e, 

where E is the approximation erroi^ : 



e = e(^\\u -TY,hiu)^p,\\^ 



Moreover, if the equality holds in pi|l ( or equivalently : the equality holds 
in p2l) ] for all u'^ G Ck) then we also have 



N -K 



il^J{U)^K) > DkE-^ - e. 



^When computing the approximation error, we consider the center of C((fci)ig/) has been 
chosen to represent all the elements coded by (P) 
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The constant Dk is given by 

Ak 



with 



B 



Dk - N-K > 



ak= e ^K{p{sy^ 

W {Cf, (r')) 



and 



"'{i'=Eie/"i'/'i,V»6/>.|<i} 



Proof. Remark first that, for any {ki)i^j E , the probabihty that 
when U ~ Eie/ Uiipi follows a uniform law in Cf^ (r'), is 

W (C((fciW)n£/,(T')) 



W (r')) 
Therefore, taking the notation of the theorem 



p (# j(c/) ^k)^ J2 k 



W (C((fc.W)n/:/,(r')) 



If (fci)ie7 is such that Ln {C{{ki)iei) n£/^ (r')) ^ 0, there exists u G C(0) 
such that w + ^Eie/ '''iV'i ^ ''--/d ('''')■ have 

/d(T5]fc.V*) < T' + fa{v) 

< t' + Mt, 

where M is given by p4|) . 
We therefore have 

F (# J(f/) . i^) < (^^i^.^^' n (r' + Mr) n p) . 

The lower bound is obtained with a similar estimation and we obtain 

W (C(0)) 
(Cf, (rO) 



Notice finally that 



#(5i!;;+°°[n/:/,(r')np) 

lim — -^--^ ■ — r- = 1. 

^^°#fe^°°'n/:/,(r'±Afr)np) 
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The proof is now a straightforward consequence of Proposition H^TUl and (|15|) . 
More precisely, taking the notations of the theorem and e > 0, we know that 
there exists T > such that, for all r < T, 

^K^ (4°;;° [ U Lf, (r' + Mr) n p) < Ak + e, 



and 



So 



En+1 

-^>r-e. 

C N + l 



(#J{{K,\ei) 



K] < 



Ak + e 

B 

/ 1 s N-K 

Ak + e { E\ "+i 



Ak - w-jf , , 
< ^E—+o{l), 

BC^^ 

where o(l) is a function of e which goes to 0, when e goes to 0. The first 
inequality of the theorem follows. 

The proof of the second inequality of the theorem is similar to one above. □ 
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